The handling systems known in the art, in particular industrial robots, for example articulated arm robots, for positioning an object gripped by means of a gripping device in a determined position and orientation in space, have internal measuring systems which can detect the position of the members of the handling system and thus provide information about the position and orientation of the gripping device in space. A distinction must be drawn in this regard between axis-related and space-related coordinate systems. The axis-related coordinate systems each relate to an axis of the robot and the respective position thereof. The kinematic chain of the individual axes and members of the robot and the respective positions thereof produces the unique location (position and orientation) of the robot tool, i.e. the gripping device, at the end of the kinematic chain. However, the location of the gripping device of an industrial robot is preferably described in a space-related manner via what is known as the TCP, the tool centre point. This is an imaginary reference point located at a suitable point on the robot tool. In order to describe what location the robot tool is to assume, the position of the TCP in space and the rotation thereof are defined. In particular by means of what is known as the Denavit-Hartenberg transformation, the robot controller is used to calculate which position the individual robot axes must assume, so that the robot tool assumes the predefined location. The location of the gripping device with its TCP preferably relates to the world coordinate system, the space coordinate system or the cell coordinate system which is for example directly or indirectly related to the base of the first axis, the base axis, the base frame or the robot base of the robot and is coupled thereto. The remaining subcoordinate systems are related to this world coordinate system, space coordinate system or cell coordinate system. It goes without saying that this world coordinate system, space coordinate system or cell coordinate system does not have to be an absolute world system; on the contrary, this system can also be subordinate to another system. The coordinate system is therefore a system which forms the superordinate reference system within the process. This system is usually coupled to the floor of the process hall, the process space or the process cell.
It is thus possible to adjust the gripping device, including the gripped object, into a determined predefined position by appropriate input on the robot controller. The gripped object is therefore positioned in space by predefining a position of the gripping device. However, this gives rise in particular to the following two problems.
On the one hand, the measuring system of conventional industrial robots which are designed for holding heavy objects is not so precise as to allow the gripping device to assume such a precise position as is required in certain production methods. The drives of industrial robots are sufficiently precise, but the measuring systems thereof are not. The measuring errors of the individual measuring members are multiplied through the kinematic chain. This results both from the measuring accuracy of the individual measuring members, in particular the angle measuring means of an articulated arm robot, and from the inevitable elasticity of the robot members.
On the other hand, the position of the gripping device and thus the location thereof in space does not yet necessarily produce the location of the object in space, as the object can usually be gripped only within a gripping tolerance. In many cases, this gripping tolerance is well above the required positioning accuracy. Thus, the gripping error, i.e. the location of the object relative to the gripping device, likewise has to be taken into consideration. Separate measuring systems which no longer pertain to the robot, in particular contactless optical measuring systems, are used for this purpose. Only these allow the object in space to be positioned in a determined location with the required accuracy.
A method for the welding-together of workpieces, in particular pressed sheet metal parts or composite metal sheets, is known from WO 2007/004983 A1 (Pettersson). The workpieces to be joined together are held by industrial robots and positioned thereby relative to one another for mutual welded connection. During the production of the welded connection, the workpieces are held by the industrial robots in the respective locations, so that the location of the parts relative to one another is maintained. The welding is carried out for example by means of a welding robot. A measuring system measures the positions of the workpieces in order to allow the workpieces to be positioned before the welding process. The measuring is carried out in particular continuously during the welding process. The described method allows the otherwise conventional, workpiece-specific moulds and workpiece receptacles, into which the workpieces have to be fixed prior to welding, which moulds and receptacles are complex to produce, to be dispensed with. The industrial robots can be used universally for differently shaped and configured workpieces, as identifying and monitoring of the workpieces and also accurate positioning of the parts relative to one another are possible as a result of the detection of the position of the workpieces by means of the measuring system. Thus, a single system can be used for different workpieces. There is therefore no need to change workpiece receptacles. According to the disclosure, the described method is suitable in particular for the welding of sheet metal parts, above all in the automotive industry. The example given of a possible measuring system is generally a laser triangulation method in which predefined points on the workpiece are measured. For this purpose, reflectors are for example attached to the workpiece. According to the description, the position of each reflector can be determined by means of a light source and a two-dimensional detector, so that the position and orientation of the workpiece can be detected by means of three such points. The precise construction of the measuring system is not described in greater detail in WO 2007/004983 A1.
U.S. Pat. No. 5,380,978 (Pryor) describes a method for positioning objects, in particular sheet metal parts, in space by means of an industrial robot. The measuring system used is in the form inter alia of cameras having an appropriate stereo base for the three-dimensional detection of the location of the object in space. The cameras are embodied in a pivotable manner, for adjusting the field of vision, and in a specific embodiment as a theodolite camera which can also have a laser distance measuring means. The described theodolite serves in this case as a precise adjusting device for the camera. Similar measuring systems are also described in U.S. Pat. No. 4,851,905 (Pryor) and U.S. Pat. No. 5,706,408 (Pryor).
A common feature of these systems and methods is the fact that the positions of a plurality of marked points on the object are determined by means of contactless photogrammetric coordinate measurements with the aid of image-processing systems.
For the contactless photogrammetric measurement of coordinates at the surface of an object in the near range, the dimensions of the object and the location thereof relative to further objects in the image is concluded from images representing the object from various perspectives, by transformation of the image data into an object coordinate system within which the object is to be measured and which is based for example on the CAD model of the object. For this purpose, the image data are processed in a data processing unit. The basis of the calculation of the coordinates is the determination of the relative camera orientations of the images involved.
It is in this case possible, as is known in the art, to record in a temporally offset manner from various perspectives the area portion to be measured of the surface of the object by means of a single camera and subsequently to process the respectively two-dimensional image data into what is known as a three-dimensional image by means of an image processing system. In this case, depth information is respectively associated with the pixels of this three-dimensional image, so that 3D image coordinates in an image coordinate system determined from the cameras and the perspectives thereof are associated with each pixel to be examined, in particular all the pixels. Different image processing methods for generating a three-dimensional image of this type from a plurality of two-dimensional images showing the same scene from different perspectives are known in the art.
It is also possible, as is likewise known in the art, to carry out, instead of the temporally offset recording of the area portion from different perspectives by means of a camera, a substantially simultaneous recording with the aid of a plurality of cameras. This has the dual advantage that three-dimensional detection of the area portion is possible without moving the camera and that detection of the respective camera orientations is dispensed with, as the cameras can have a fixed relative orientation to and distance from one another.
Different 3D image recording means which are composed substantially of two or three cameras, which are accommodated so as to be set apart from one another, i.e. having a stereo base, in a common housing so as to be securely coupled to one another for recording a scene from an in each case different, but fixed relative perspective, are known in the art. As the recorded area portion does not necessarily have characteristic image features allowing the images to be electronically processed, markings can be applied to the area portion. These markings can be produced by means of a structured light beam, in particular a laser beam, which is projected from the 3D image recording unit onto the area portion and projects for example an optical grating or an optical marking cross. In many cases, 3D image recording units of this type also contain an image processing means which derives a three-dimensional image from the plurality of images, recorded substantially at the same time, of different perspectives.
Examples of 3D image recording units of this type are the image recording systems known under the brand names “Optigo” and “OptiCell” from the company “CogniTens”, which contain three cameras arranged in an isosceles triangle, and also the “Advent” system from the company “ActiCM” with two high-resolution CCD cameras arranged next to each other and also a projector for projecting structured light onto the portion to be recorded.
The coordinates of recorded image elements to be measured are generally determined by means of referenced markings within the image, from which markings the actual 3D coordinate measurement takes place. In this case, the image coordinate system, which relates to the recorded three-dimensional image and is thus related to the 3D image recording unit, is transformed into the object coordinate system within which the object is to be measured and which is based for example on the CAD model of the object. The transformation takes place on the basis of recorded reference markings, the positions of which in the object coordinate system are known. Accuracies of less than 0.5 millimeter are achieved in this case with the 3D image recording units known in the art.
3D scanning systems, in particular in the form of 3D laser scanners, which carry out a depth scan within an area region and generate a point cloud, are also known. In this case, a distinction must be drawn between serial systems, in which a laser beam scans an area line by line, parallel systems, in which a scan line is fanned out over an area, and fully parallel systems, what are known as RIMs or range imaging systems, which simultaneously scan a large number of points within an area region and thus carry out a depth recording of the area region. A common feature of all these systems is generally the fact that the depth scanning takes place by means of at least one distance measuring laser beam which is in particular moved over the area. In particular serial systems of this type are widespread and commercially available for example under the product names “Leica HDS 6000”, “Leica ScanStation 2”, “Trimble GX 3D Scanner”, “Zoller+Fröhlich IMAGER 5003” and “Zoller+Fröhlich IMAGER 5006”.
A problem of each 3D image recording unit is the recording range within which it is possible to record images with the required resolution, this range being limited due to the design. In the three-dimensional detection of relatively large objects, it is therefore inevitable to make a plurality of individual three-dimensional recordings from different positions and orientations of the 3D image recording unit. This large number of smaller image recordings are subsequently joined together to form a larger three-dimensional total image by means of compensation of overlapping image regions and with the aid of markings within the recorded area portion. Different methods for achieving this object are known in the art. A general problem with these methods consists in the fact that the individual three-dimensional images which are to be joined together to form a larger image have to have a region of overlap. The image-processing systems do not allow the position of the 3D image recording unit to be discretely varied from a first area portion having at least one reference point to a second area portion which is set apart from the first area portion and does not contain any reference points, if no further images connecting the two area portions were recorded. It is therefore necessary to carry out a large number of intermediate image recordings in order to optically connect the two set-apart area portions to be measured and to allow coherent image processing. The recording of a large number of three-dimensional images having no direct measuring content slows down the measuring method as a whole and uses up memory and computing resources. Furthermore, the coordinate measurements, which inevitably contain minor measuring errors, within the image recording have a drastic effect on measuring accuracy during the composition of the large number of images, in particular in the case of remote reference points.
The use of a large number of reference points having known positions in the object coordinate system is thus inevitable on account of the limited field of vision of the cameras. An advantage of the purely photogrammetric systems described consists in the fact that the absolute position and orientation of the individual cameras of the 3D image recording unit in the object coordinate system does not have to be determined, since the absolute position of the recorded pixels is determined from the knowledge of the position of the likewise recorded reference points in the image, the orientation of the cameras relative to one another and also the relative positions, calculated via triangulation, of the points to be measured relative to the reference points in the image. The measuring system may thus be limited to image-calibrated cameras, the position of which relative to one another is known, and an image processing means.
A drawback of all these systems consists in the fact that an adjustment of the field of vision, by either pivoting or varying the position of the cameras or the object to be measured, is often inevitable on account of the limited field of vision of the cameras and the limited image resolution. This is the case above all when measuring relatively large objects to be measured with high precision, as a determined distance of the cameras from the object may not be exceeded, on account of the limited image resolution, in order to adhere to the required measuring accuracy, although the field of vision of the camera allows only a part of the object to be recorded at such proximity to the object. It is thus necessary either to use a large number of reference points, so that in each image recording a corresponding number of reference points, preferably at least three reference points, lies in the field of vision, or to draw on the positions of object points which have already been determined beforehand, in particular markings on the object.
In this case, a plurality of individual three-dimensional recordings are, as described above, made from different positions and orientations of the 3D image recording unit. This large number of smaller image recordings are subsequently joined together to form a larger three-dimensional total image by means of compensation of overlapping image regions and with the aid of markings within the recorded area portion. This is time-consuming and requires the use of markings which cannot be measured per se.
Also known in the art are measuring systems and methods in which the 3D image recording unit is carried by the head of an industrial robot or a gantry coordinate measuring machine and is adjustable. Precise detection of the position of the 3D image recording unit at the required accuracy, which is equivalent to the image recording accuracy, is not possible on account of the high weight of a high-quality and high-resolution 3D image recording unit, which is in some cases more than 10 kilograms, as this would require such a stable construction of the handling system that the range of use of the 3D image recording unit would be limited to stationary systems. Industrial robots are unsuitable for external referencing on account of their comparatively low measuring accuracy which is much lower than that of a precise 3D image recording unit. Gantry coordinate measuring machines are, for their part, not designed for carrying heavy loads and, under high mechanical loading, do not provide any measured results which can be used for referencing. For this reason, the product measured values, which may be supplied by the handling system and might provide information about the absolute and/or relative position of the 3D image recording unit, cannot be adduced for referencing the image recordings, in particular a plurality of three-dimensional image recordings of different, non-coherent area portions.
Although the described measuring systems are also suitable for the high-precision positioning of objects in space by means of handling systems and are also used for this purpose, the systems known previously in the art have numerous drawbacks. On account of the above-described measuring method, which takes place substantially purely via image processing, the methods are relatively time-consuming and require the detection of reference or auxiliary markings which cannot be measured per se. On account of the limited field of vision of the cameras, the 3D image recording units are arranged usually in direct process proximity, generally on a robot arm or at a short distance from the object. Owing to the process proximity associated therewith, the 3D imaging unit is exposed to any particles and thermal influences produced by the process—for example during welding. Likewise on account of the process proximity, further handling systems have to be adapted to the handling system of the 3D image recording unit in order to avoid collisions. Moving the 3D image recording unit and the new referencing associated therewith are comparatively time-consuming and slow down the process sequence as a whole. As three-dimensional image detection requires the relative location of the plurality of cameras to be known at all times, independent orientation of the cameras is avoided. Instead, the cameras are preferably mechanically coupled to one another. As process-remote positioning of the 3D image recording unit requires the cameras to be set correspondingly far apart from one another in order to allow a sufficient stereo base for three-dimensional image detection, the cameras can in this case no longer be mechanically coupled. In the past, a process-remote arrangement of the 3D image recording unit has thus been dispensed with altogether. The two objectives, on the one hand a high-precision, contactless 3D measuring system having an accuracy of preferably less than 0.1 millimeter for the high-precision positioning of objects by means of industrial robots and on the other hand a measuring system which is not directly exposed to the process, can be handled in a flexible manner and can in particular be freely positioned, are thus a conflict of objectives that has not yet been sufficiently solved in the field of the industrial positioning of objects by means of industrial robots.